Bayesian Brittleness: Why no Bayesian model is “good enough”

Although it is known that Bayesian estimators may fail to converge or may con-verge towards the wrong answer (i.e. be inconsistent) if the probability space is not finite or if the model is misspecified (i.e. the data-generating distribution does not belong to the family parametrized by the model), it is also a popular belief that a “good ” or “close ” enough model should have good convergence properties. This paper incorporates Bayesian priors into the Optimal Uncertainty Quantifica-tion (OUQ) framework [86] and in doing so reveals extreme brittleness in Bayesian inference. These brittleness results demonstrate that, contrary to popular belief, there is no such thing as a “close enough ” model in Bayesian inference in the follow-ing sense: we derive optimal lower and upper bounds on posterior values obtained from models that exactly capture an arbitrarily large (but finite) number of finite-dimensional marginals of the data-generating distribution and/or that are arbitrarily close to the data-generating distribution in the Prokhorov or total variation metrics; these bounds show that such models may still make the largest possible predictio